Papers with language generation model
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models are based on sequential decoding or serialisation of structured data into sequence. |
| Approach: | They propose a model that combines sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. |
| Outcome: | The proposed model produces fluent translations with better reordering than previous models. |
Informed Sampling for Diversity in Concept-to-Text NLG (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to encourage lexical diversity for language generation tasks produce repetitive outputs, but this often comes at a cost to the perceived fluency and adequacy of the output. |
| Approach: | They propose to augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output. |
| Outcome: | The proposed method achieves a high level of diversity with minimal effect on the output’s fluency and adequacy. |
Textual Data Augmentation for Efficient Active Learning on Tiny Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing active learning approaches for textual data are limited due to the complexity of language. |
| Approach: | They propose an approach where guided outputs of a language generation model can be enhanced through an active learning process. |
| Outcome: | The proposed approach achieves performance increases of 3% and 5% on TREC-6 and SST-2 datasets compared with NGDG, which does not optimize for a reward function. |
Extrapolating Multilingual Understanding Models as Multilingual Generators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual understanding models are not capable of generating high-quality text compared with decoder-based causal language models. |
| Approach: | They propose a method to adapt a multilingual encoder to a language generator with a small number of additional parameters. |
| Outcome: | The proposed approach outperforms initialization-based methods with 9.4 BLEU on machine translation, 8.1 Rouge-L on question generation, and 5.5 METEOR on story generation. |